Visual head tracking and slaving for visual telepresence

This paper describes a system for visual telepresence that slaves the motion of an electromechanical stereo head/eye platform to that of an operator’s head. Features on the operator’s head are detected using an external static camera and tracked over time, using three or more features to find pose (orientation and translation) with a novel algorithm which avoids explicit recovery of the features’ 3D positions. The recovered pose is then transformed into demands for the pan and elevation axes of the head platform. The process of capturing images, tracking features, recovering pose, and generating demands runs at 50Hz with a latency of 23ms, with the controller and platform typically taking another 20ms to satisfy a demand. Experiments show that the dynamic performance of the entire system, including head platform, is sufficient to copy head movements of up to s .